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Abstract. We introduce an analytically solvable model of two-dimensional continuous 
attractor neural networks (CANNs). The synaptic input and the neuronal response form 
Gaussian bumps in the absence of external stimuli, and enable the network to track external 
stimuli by its translational displacement in the two-dimensional space. Basis functions of the 
two-dimensional quantum harmonic oscillator in polar coordinates are introduced to describe 
the distortion modes of the Gaussian bump. The perturbative method is applied to analyze its 
dynamics. Testing the method by considering the network behavior when the external stimulus 
abruptly changes its position, we obtain results of the reaction time and the amplitudes of 
various distortion modes, with excellent agreement with simulation results. 



1. Introduction 

Continuous attractor neural networks (CANNs) are very useful models for describing the 
encoding of continuous stimuli in neural systems [1—5]. The encoded stimuli can either be 
some simple features of objects, such as their orientations [6], moving directions [7] and spatial 
locations [8], or some complicated rules that underly the categorization of objects [9]. Compared 
with other attractor models, CANNs have the distinctive feature that they hold a family of 
stationary states which can be translated into each other without the need to overcome energy 
barriers. In the continuum limit, these stationary states form a continuous manifold in which 
the system is neutrally stable, and the network state can translate easily when the external 
stimulus changes continuously. Beyond pure memory retrieval, this large-scale structure of the 
state space endows the neural system with a tracking capacity. 

To construct a model for CANN, the key is that the neuronal interactions should be properly 
balanced in excitation and inhibition (e.g., of the Mexican-hat shape) and be translationally 
invariant. The former enables the network to have a local persistent bump solution and the latter 
ensures that the network has a continuous family of such solutions. Although mathematically it 
is possible to construct a CANN of dimensionality larger than two, the research on CANNs has 
so far been mainly focused on one or two dimensional case. This is because in the cortex, neurons 
are essentially distributed in a two-dimensional sheet. To maintain a CANN of dimensionality 
larger than two, it is difficult to wire neurons in a two-dimensional space without their signals 
interfering with each other. To encode continuous features of high dimensionality, the brain may 
employ layers of neurons to combine low-dimensional CANNs hierarchically. 



The tracking dynamics of a CANN has been theoretically investigated by several authors 
in the literature [5,6, 10-12]. These studies have demonstrated that a CANN has the capacity 
of tracking a moving stimulus continuously and that this tracking behavior can describe many 
brain functions well. Despite these successes, however, detailed rigorous analysis of tracking 
behaviors of a CANN is still lacking. In a recent work [13, 14], the authors have developed a 
perturbative approach to elucidate the tracking performance of a one-dimensional CANN clearly. 
Because of the bump shape of the stationary states of the network, we used the wave functions of 
the quantum harmonic oscillator as the basis to decompose the network dynamics into different 
motion modes. These modes have clear physical meanings, corresponding to distortions in the 
amplitude, position, width or skewness of the network state. Due to the neutral stability of 
network states, the dynamics of a CANN is typically dominated by a few motion modes, with 
their contributions determined by the corresponding eigenvalues. We therefore can project the 
network dynamics on these dominating modes and simplify the network dynamics significantly. 
In this study, we extend the perturbative approach to a two-dimensional CANN. The two- 
dimensional CANN has much richer dynamics and distortion patterns than the one-dimensional 
case [15-17]. To elucidate the effect of different distortion patterns on the network dynamics 
clearly, we develop the perturbative approach in both rectangular and polar coordinates. To 
test our method, we study the tracking performance of the network when the external stimulus 
position experiences an abrupt change. Simulation results confirm that our method works very 
well. 



2. The Model 

We consider a two-dimensional neural network coding the stimulus x = (x\, x%), with N neurons 
distributed over this space. For simplicity, the neurons are assumed to be uniformly distributed 
in the space. Considering the common case that the range of possible values of the stimulus being 
much larger than the range of neuronal interactions, we can effectively take — oo < xi, X2 < oo. 
The dynamics of the synaptic input C/(x, t) and neuronal response r(x, t) is given by 



dU(x, t) 



dt 

r(x,f) = 
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where J(x, x') is the translationally invariant coupling function defined by 
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a is the tunning width of the neural network, k is the global inhibition, and p is the density of 
neurons over the space. When J ext = and < k < k c = A 2 p/(32ira 2 ), we have the steady 
solutions, or stationary states, given by (see figure 1) 
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whereto = [l + {l-k/k c ) l l 2 ]A/{%-Ka 2 k) and r = [1 + (1 - k/k c ) l l 2 ]/{Ana 2 kp). It is notable that 
Eqs. (4) and (5) are valid for any z. For simplicity, we consider I ext = alio exp[— |x — zo| 2 / (4a 2 )], 
where a is the strength of the stimulus. Thanks to the translational invariance of the coupling 



function, the stationary state solution can be peaked at any point in the space. In this paper, we 
consider the network response to a stimulus abruptly changed from (x±, X2) = (0, 0) to (zoi, ^02) 
at t = 0. As shown in the simulation result in figure 2, the synaptic input can track the change. 




Figure 1. A stationary state 
U(x,0). Parameters: N = 40 x 40, 
a = 0.5, k = 0.5, r = 1 and 
p = N/(2ir) 2 . 



Figure 2. The synaptic input 
in the space due to the abruptly 
changed stimulus. Parameters: 
a = 0.05, z (t < 0) = 0, z (t > 
0) = (1,1) and the rest are the same 
as figure 1. 



3. Solution to the Model 

Under the driving of an external stimulus, the network state (i.e, the bump) moves from its initial 
position to the target one, with its shape distorted during the tracking process. Thus, to describe 
the tracking performance of a CANN, the key is to know the distortion patterns and their effects 
on the network dynamics. We denote the the state distortion to be 5U (x, t) = U (x, t) — f/(x|z), 
whose dynamics is given by linearizing Eq.(l) at f7(x|z) [13, 14], 
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where the interaction kernel F(x, x'|z) is 
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where B = 1 + kpf™ f^° oo U(-x'\z) 2 dx' is a constant. The network dynamics is determined by 
the eigenfunctions and eigenvalues of the kernel -F(x, x'|z). To compute them, we choose the 
eigenfunctions of the quantum harmonic oscillator as the basis. While the results are more clearly 
presented using basis functions in polar coordinates, the analysis is more conveniently done in 
the rectangular coordinates. Hence we will describe the basis functions in both coordinates. 



3.1. Using Basis Functions in Rectangular Coordinates 
Under the rectangular coordinates, the basis functions are 
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where H n (x) is the n th order Hermite Polynomial [18]. By the completeness of the basis 
functions, we have 

an,m{t)u n , m (x.\z(t)). (9) 

n,m 

Thanks to the orthonormality of the basis functions, we have derived [13, 14] 
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where I n>m is the projection of I cxt onto Un m , given by 
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and F n ,m,n',m'i the interaction matrix, is given by 
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The center of mass is given by the self-consistent condition z{t) = J J dxU(x, t)x/ J J dxU(x,t). 
If the external stimulus is symmetric with respect to the x-axis, then dz 2 /dt = and 
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The eigenvalues of J 7 are Ao,o = 1 — (1 — k/k c ) l l 2 , A nj o = A ra for n / 0, Ao im = A m for 
m / 0, A rai m = A n A m for n / and m / 0, where A n = 2/2 ra . From this result, one 
can conclude that, if the stimulus is absent, all modes of distortion will decay exponentially 
in time, except for the eigenfunctions u^o and uo,i, whose eigenvalues are 1. To prove this, 
we define (x|z) to be the right eigenfunctions of T. Then we may express 6U{x,t) in 
Eq. (6) as 8U(x,t) = Y^ n m SU njm (z, t)v^ m (x.\z) . Using the orthonormality of the left and right 
eigenfunctions of F(x, x'|z), we have 



/oo poo 
/ dx5U(x,t)v^ m (x\z). 
-oo J — CO 



(16) 



Assume that the motion of the bump is slow, so that dz/dt becomes negligible in Eq. (6); as we 
shall see, this assumption is valid as long as the external stimulus is sufficiently weak. Then, 
the projection of Eq. (6) on the eigenfunctions become 



T^5U ntm (z,t) = (A„, m - l)SU ntm (z,t). 
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Hence, 
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where 5U ntin (z,0) is the initial value of the projection. u± t o and uo,i correspond to the 
trackability of the synaptic input as well as the neuronal response, because ui t o ~ dU /dx\ and 
^o,i ~ dU/dx2- uifl and uo,i are the modes of the position shift of the synaptic input. Thus, it 
guarantees the stability of the stationary solution and trackability of the synaptic input. 

We are now ready to find the tracking solution to the stimulus abruptly changed from (0, 0) 
to (zqi, 0) at t = 0. Neglecting the depdendence on all a„ iTn terms in Eq. (15), we have 
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which is consistent with the result obtained from the one-dimensional case [13, 14]. This 
approximation is useful when \z — zq\ is small and the stimulus is weak. It also shows that 
the tracking behavior is similar to the one-dimensional case. 



3.2. Using Basis Functions in Polar Coordinates 

Since the system is also rotationally invariant, the analysis can proceed by using the 
eigenfunctions with polar coordinates. The eigenfunctions of quantum harmonic oscillators are 
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where i = \J — 1, and I, j are the radial and angular quantum numbers respectively. Decomposing 
the distortion term, we have SU = Yli jbl,j(t)i/>i,j- The matrix elements of the transformation 



matrix are 
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Similar to Eq. (10), there is an interaction kernel Q that represents the interaction between 
different ipi j, which can be obtained by Q = T~ X TT. The first few eigenfunctions of Q are 
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where the indices I and j of ^>ij represent the highest basis function it contains. Their eigenvalues 
are Aoo, 1, 1/2, 1/2, 1/4, and 1/4 respectively. 

As shown in Figs. 3 to 8, the eigenfunctions are symmetric with respect to the origin. The 
eigenfunctions correspond to different modes of the distortion of the synaptic input during the 



motion. \&oo corresponds to the change in height. It can describe, say, the reduction of the bump 
height during the process to catch up with the new position of the stimulus, as shown in figure 
2. ±1 can describe the movement of the bump towards the preferred positions. \E f 2,o describes 
not only changes in the height of the bump, but also changes in the width of the bump during 
the motion. ^3 : ±i describes the skewing of the bump due to the stimulus and other modes. 
While the above distortion modes are apparently extensions of those in the one-dimension case, 
the modes ^2,±2 and ^,±3 are unique to the two-dimensional case. The former corresponds to 
an elliptical distortion of the bump shape, and the latter to a three-fold distortion. 





Figure 3. Real part of the eigenfunction 

*0,0- 



Figure 4. Real part of the eigenfunction 





Figure 5. Real part of the eigenfunction 

*2,0- 



Figure 6. Real part of the eigenfunction 



By using the transformation matrix in Eq. (21), Eq. (10) can be transformed from rectangular 
to polar coordinates up to arbitrary order. For the perturbation up to / = 2, and for external 
stimuli symmetric with respect to the x-axis, we have 
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Figure 7. Real part of the eigenfunction 
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4. Simulation Experiments 

In the simulation experiments, the number of neurons is N = N x x N y , and the range of (x±, X2) 
is — 7T < x±,X2 < 7r. The boundary condition is periodic. 

^.i. Reaction Time to an Abrupt Change of the Stimulus 

In this experiment, the stimulus is centered at (0, 0) until the synaptic input U(x, t) becomes 
steady. At t = 0, the stimulus abruptly changes from (0,0) to zq = (zoi,0), and we observe the 
dependence of the reaction time on the distance zoi- Then, the bump will track the stimulus, 
as shown in figure 2. The reaction time is defined by the time needed to have \z(t) — zq\ < 0, 
where is the threshold. This threshold is necessary in this experiment, because the motion of 
z(t) will become very slow when it approaches the stimulus, as implied by Eq. (19). Also, the 
assumption is reasonable because in real biological systems, we do not need to have z(t) = zo(t) 
to make decisions. 



Figure 8. Real part of the eigenfunction 
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Figure 9. The reaction time T for 
the syanptic input to catch up the 
stimulus position change from to 
zq\. Parameters: N = N x x N y = 
40 x 40, = tt^/2/N, a = 0.05 and 
the rest are the same as figure 1. 



As shown in figure 9, the prediction given by Eq. (19) works well only when the change in 
the position of the stimulus is small, while the I = 3 perturbation works well up to |zo| = 2. 
The prediction of the I = 10 perturbation is the best among the three. From this result, one 
can state that, when the position change of the stimulus is small, only ip^j with small I will be 
activated. However, when the change in stimulus position is larger, higher order distortions are 
activated. This is reasonable, because, for smaller I, the distortions are concentrated around 
z(i), but if the stimulus is far away from z(i), the tail part of the bump will be distorted first, 
leading to higher order distortions. 



4-2. Amplitudes of the Basis Function Distortion Modes 

From Eqs. (10) and (21), bij can be predicted by the projections of SU onto the basis functions 
tpij- The experimental settings are the same as above, but zq was fixed to be zq = (2,0) in 
polar coordinates. 

As shown in figure 10, the predicted bij's agree with the simulation results well. It confirms 
that the perturbative method can also predict the motion of the synaptic input in detail. &o,o 
indicates that the height drops from its initial value after the stimulus is shifted. The ^o,o 
component of the distortion is reduced by the inhibition. It approaches roughly, as if there 
were no external stimulus, and there is even a slight overshoot. Afterwards, the distortion 
relaxes smoothly to the equilibrium value when it approaches the shifted position of the stimulus. 
Similarly, the amplitude of the ipi ±i components falls abruptly to a negative value initially. This 
is due to the tail of the bump being pulled by the newly positioned stimulus, causing the peak 
to lag behind the center of mass. Afterwards, it relaxes smoothly to 0. The initial change in 
62,0 indicates an increase in width along the direction of the stimulus, and that of &2,±2 signals 
a cigar-shaped distortion when the bump is being pulled by the stimulus. The amplitude &3,±i 
describes the skewness of the bump, and 63^3 describes the bump being distorted when its tip 
is pulled by the stimulus, with the posterior part lagging in motion, causing a triangular-shaped 
distortion. 
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Figure 10. The experimentally projected bij and the predicted bij using I = 10 perturbation. 
Solid lines: corresponding predictions. Circles: simulation results. Parameters: same as figure 
9. 



5. Conclusion and Discussion 

In this paper, a perturbative method to deal with continuous attractor neural networks is 
presented in two-dimensional space. We have introduced a simple solvable model to demonstrate 
how to use perturbative method to analyze the dynamics of distortions and synaptic input. Since 
the coupling factors are defined to be translational invariant, a family of stationary states can 
be sustained anywhere in the preferred stimulus space. Furthermore, the synaptic input is able 
to track the stimulus in the space. By studying the dynamics, one can deduce the tracking time 
and the distortions of the synaptic input. 

For one-dimensional continuous attractor neural networks, the tracking speed can be roughly 
approximated by 

dz a 
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which is similar to Eq. (19). Although they are rough approximations, the basic properties of 
tracking in one and two dimensions are similar. The key predictions in the one-dimensional 
case, the maximum trackable speed [11, 13] and the lag behind a continuously moving stimulus, 
are also applicable in two dimensions. These similarities arise from the rotational invariance 
of the interaction kernel and the bump in the limit of weak stimulus, since the description 
of the dynamics by the translational mode in sufficient. Note, However, for the stronger 
stimuli, the dynamics is richer in the two-dimensional case, since distortion modes unique to 
the two-dimensional case need to be considered; examples of cigar-shaped and triangular-shaped 
distortions are shown in figure 10. 

For this particular model, the eigenfunctions of the interaction matrix in polar coordinates 
are also studied. By Eq. (21), the interaction matrix Q can be obtained. However, due to 
complications in the calculation, we can only calculate it term by term. Since the matrix is 
upper triangular, the eigenvalues are the diagonal entries, \ n ,m- As the eigenvalues are at 
most 1, one can show that the synaptic input has a stable Gaussian form. The eigenfunctions 
corresponding to eigenvalue 1 represent the positional shift. As studied above, different modes 
of distortion correspond to different kinds of distortion. For example, the component of ^o,o 
corresponds to the change in height, while ^2,0 represents the change in width. 

Concerning the robustness of the method, we remark that it can be applied to other types of 
networks with tracking behavior. A common example is the continuous attractor neural network 
with the Mexican hat interaction. Using the basis functions of the two-dimensional quantum 
harmonic oscillator, we can obtain the matrix elements of the interaction kernel numerically, 
although elegant expressions such as those obtained here may not be available. Perturbation 
dynamics can then be worked out analogously. This proposed extension can be able to address 
a recent issue of interest, namely, the instability of bumps and rings in a two-dimensional neural 
field of Amari type [19]. Meanwhile, we note in passing that the present model with a quadratic 
response and a global inhibition does not suffer from the stability problem. 
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